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Abstract 

The Container Relocation Problem (CRP) is concerned with finding a sequence of moves of 
containers that minimizes the number of relocations needed to retrieve all containers respecting 
a given order of retrieval. While the problem is known to be NP-hard, there is much evidence 
that certain algorithms (such as the A* search [2^) and heuristics perform reasonably well on 
many instances of the problem. 

In this paper, we first focus on the A* search algorithm, and analyze lower and upper bounds 
that are easy to compute and can be used to prune nodes and also to determine the gap 
between the solution found and the optimum. Our analysis sheds light on which bounds result 
in fast computation within a given approximation gap. We also present extensive simulation 
results that improve upon our theoretical analysis, and further show that our method finds the 
optimum solution on most instances of medium-size bays. On the “hard” instances, our method 
finds an approximate solution with a small gap and within a time frame that is fast for practical 
applications. We also study the average-case asymptotic behavior of the CRP where the number 
of columns grows. We calculate the expected number of relocations in the limit, and show that 
the optimum number of relocations converges to a simple and intuitive lower-bound. This gives 
strong evidence that the CRP is “easier” in large instances, and heuristics such as finds near 
optimal solution. 

We further study the CRP with incomplete information by relaxing the assumption that the 
order of retrieval of all containers are initially known. This assumption is particularly unrealistic 
in ports without an appointment system. In our model, we assume that the retrieval order of 
a subset of containers is known initially and the retrieval order of the remaining containers is 
observed later at a given specific time. Before this time, we assume a probabilistic distribution 
on the retrieval order of unknown containers. We extend the A* algorithm and combine it with 
sampling technique to solve this two-stage stochastic optimization problem. We show that our 
algorithm is fast and the error due to sampling and pruning the nodes is reasonably small. Using 
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our framework, we study the value of information and its timing effect on the average number 
of relocations. 


1 Introduction 

With the growth in international container shipping in maritime ports, there has been an increasing 
interest in improving the operations efficiency in container terminals on the sea side and land side. 
The operations on the sea side include loading export containers on the vessels or discharging 
import containers from vessels and loading them onto internal trucks. The import containers are 
then transferred to the land side and are stacked in the storage area. These containers are later 
retrieved and delivered to external trucks to be distributed in the city. 

Due to limited space in the storage area, containers are stacked in tiers on top of each other. As 
shown in Figure [H several columns of containers in a row create a bay of containers. If a container 
that needs to be retrieved {target container) is not located at a top most tier and is covered by 
other containers, the blocking containers must be relocated to another slot. As a result, during the 
retrieval process, one or more relocation moves are performed by the yard cranes. Such relocations 
are costly for the port operators and result in delay in the retrieval process. Thus, reducing the 
number of relocations is one of the main challenges faced by port operators. Finding the sequence 
of moves that minimizes the number of relocations while retrieving containers from a bay in a pre¬ 
defined order is referred to as the Container Relocation Problem (CRP) or the Block Relocation 
Problem (BRP). A common assumption of the CRP is that only the containers that are blocking 
the target container can be relocated. We refer to the CRP with this setting as the restricted CRP. 


Slot 



Column 1 


Figure 1: Illustration of a bay of containers in storage area 


The CRP has been widely studied in the literature and has been shown to be NP-hard [^. A 
few papers have developed mathematical models for the CRP and have solved small-sized instances 
of the problem exact! . In other papers, different heuristics are designed and tested on 

real-sized instances ( ). Many of these heuristics are fast and shown to be empirically 

close to optimal for small-sized instances. However, none of them have provable guarantees and their 
performance on large-sized instances cannot be well assessed. Among the optimization methods 
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that provide exact solution for the CRP, the A* algorithm (a branch and bound framework) has 
shown to be very effective and able to solve large instances of the problem in a time frame that is 
suitable for practical application 20|, 2^, llol. 

In this paper, we apply the A* algorithm ([20|, 2^, 1^) to the CRP, and analyze various upper 
and lower bounds. First, we study the properties of a counting lower bound that simply counts the 
number of blocking containers in the initial configuration (introduced in 0]). We show that this 
bound is non-decreasing on each path of the A* tree, implying that the guaranteed optimality gap 
given by the A* algorithm may only be improved as more levels of the tree are constructed (see 
Proposition 12.2p . Moreover, we analyze a generalized family of look-ahead lower bounds that also 
counts the future relocation moves due to unavoidable blocking (introduced in [2^), and we prove 
that this family provide tighter lower-bounds that are still monotone in each path (see Proposition 
Ea and 12.51 and Figure fTOj) . Our result implies that using look-ahead lower bounds (with a slightly 
higher computation time compared to the counting bound) can decrease the number of branching 
needed to achieve a desired optimality gap. 


We also study several properties of the H heuristic presented in which indicates that this 
heuristic serves well as an upper bound. In particular, we prove that the A* algorithm terminates 
after a certain level when the H heuristic is used combined with the appropriate lower bound (see 
Proposition 12.8112.71 and l2.9l) . Our numerical simulations show that, the algorithm typically finds 
the optimal solution much faster than what is suggested by the theoretical results (see Figures [5] 
and [9]). We also use the idea of branching to design a heuristic called the Tree-Heuristic (TH-L, 
see Algorithm [2]) . We use randomly generated instances to benchmark several existing heuristics 
and the new Tree Heuristic against the optimal solution given by the A* algorithm. We show that 
TH-L has the best performance (smallest optimality gap) on average (see Table E]). 

Building upon the previous results, we perform an average case analysis of the CRP ir^a bay 
with finite height. First we compute the expected value of the counting lower bound 


see 


Proposition 13.ip for an evenly distributed bay. Then we provide an asymptotic result and prove 
that as the number of columns grows, in expectation, the optimum number of relocations converges 
to the counting lower bound (see Theorem 13.31) . The intuition behind this result is that as the 
number of columns grows, the probability of finding a good column for a blocking container (i.e., 
a column where the blocking container will never be relocated again) approaches 1. In that case, 
the optimal number of relocations is essentially the number of blocking containers in the initial 
bay. Furthermore, we strengthen this result by showing experimentally that the expected number 
of relocations performed by heuristic H (Qj) also converges to the counting lower bound (see Figure 
6aP . This further highlights the effectiveness of both heuristics H and TH-L in the case of large 
instances. 

A critical assumption of the CRP is that the retrieval order of all containers in the bay is 
considered to be known in advance. However, in practice, such information is not available far in 
advance and the exact departure order of containers becomes known only as their retrieval times 
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approach. Here we relax this assumption in the following way: We consider a 2-stage setting, 
where the retrieval order of a subset of containers is known initially and the retrieval order of 
the remaining containers is revealed all at once at a later time. We refer to this problem as the 
CRP with incomplete information. Before the entire information becomes available, we assume 
a probabilistic distribution on the unknown retrieval orders. We develop a 2-stage approximate 
stochastic optimization algorithm, called ASA*, and use it to solve medium-sized instances of the 
CRP with incomplete information. This algorithm is an extension of the A* algorithm combined 
with sampling and pruning techniques based on the bounds defined earlier. We give theoretical 
bounds on the approximation error incurred by this algorithm (see Propositions 14.1114.21 and 14.3p . 
Furthermore, we introduce a myopic heuristic, which is fast and performs well on average compared 
to the ASA* algorithm (see Figure d^ . We use the heuristic to solve large instances of the problem 
and show through experiments that the ratio of expected number of relocations in the incomplete 
information setting to that of complete information converges to a constant, as the bay gets larger 
(see Figure [13]). Moreover, we use the ASA* algorithm and the myopic heuristic to assess the value 
of information (see Figure [T^ . Our experiments show that when the retrieval order of 50%-75% of 
containers is known at the beginning, the loss from the missing information is negligible and the 
average number of relocations is very close to that of the CRP with full information. 

Our algorithms can serve as a decision support tool for port operators as well as an analytical 
tool to draw insights on managerial decisions such as determining the size/capacity of the bays 
and assessing the value of information. This issue of lack of information is particularly critical as 
it captures port operations more realistically. Understanding the value of information could help 
port operators design efficient appointment systems for trucks. 


1.1 Literature Review 




Stahlbock and VoB[l5l] provide a general survey on port operations literature. For a recent general 
review and classification survey of the existing literature on the CRP as well as the stacking problem, 




we refer the reader to Lehnfeld and Knust 

Many heuristics have been developed for the CRP. Kim and Hong suggest a decision rule that 
uses an estimate of expected number of additional relocations for a column Q]. Caserta et al. El 
propose a binary description of configurations and use it in a four steps heuristic. Lee and Lee [11] 
consider an extended objective incorporating working time of cranes in a three phases heuristic. 
Caserta et al. derive a meta-heuristic based on the corridor method, which again optimizes 
over a restricted set of moves and Caserta et al. in Qj present a heuristic that is defined later in 


this paper. Unliiyurt and Aydm 17j propose a branch and bound approach with several heuristics 
based on this idea. Foster and Bortfeld J7| use a tree search procedure to improve a greedy initial 
solution. Finally, Petering and Hussein [l^ describe a look-ahead heuristic that does not make the 
common assumption of the restricted CRP. In our experimental result, we use randomly generated 
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instances to compare the performance of these heuristics (and our new TH-L heuristic) to that of 
the optimal solution given by the A* algorithm. We show that on average our TH-L outperforms 
the existing heuristics. 

In the mathematical programming formulations that insure optimality on the entire feasible set 
of moves, the first binary formulation was suggested by Kim and Hong [^. Petering and Hussein [l^ 
propose another formulation which is more tractable but cannot solve real-sized instances efficiently. 


Caserta et al. derive a different Integer Program (IP). Borjian et al. develop an IP formulation 
which allows for incorporating a time component into the model. This model is used to jointly 
optimize the number of relocations and total service delay. In all these IP formulations, due to the 
combinatorial nature of the problem, the number of variables and constraints dramatically increases 
as the size of the bay grows, and the IP cannot be solved for large instances. A way to bypass 
this problem has been to look at a new type of algorithm called A*. It was first introduced to this 
problem by Zhang et al. 20], studied by Zhu et al. 22], and Tanaka and Takii [l^ proposed a new 
lower bound for the algorithm. Our paper contributes to this line of research (applying A* to CRP) 
by theoretically analyzing several lower and upper bounds, also by computing (both theoretically 
and via simulation) how the optimality gap behaves as the number of branches increases. 

Average case analysis of CRP is fairly new. The only other paper in this direction is the recent 
paper by Olsen and Gross [3]. They also provide a probabilistic analysis of the asymptotic 
CRP when both the number of columns and tiers grow to infinity. They show that there exists a 
polynomial time algorithm that solves this problem close to optimality with high probability. Our 
model departs from theirs in two (related) aspects: (i) We keep the maximum height (number of 
tiers) a constant whereas in [l^ the height also grows. Our assumption is motivated by the fact 
that the maximum height is limited by the crane height, and it cannot grow arbitrarily, (ii) We 
assume the ratio of the number of containers initially in the bay to the bay size stays constant (i.e., 
the bay is almost full at the beginning). On the other hand, in [l^, the ratio of the number of 
containers initially in the bay to the bay size decreases (and it approaches zero) as the number of 
columns grows. In other words, in the model of [l^, in large bays, the bay is under-utilized. 

Finally, we mention that another major challenge in port optimization is uncertainty (together 
with the value of information) which has not been considered in the literature until very recently. 


Zhao and Goodchild 21[ model uncertainty in truck arrivals at the terminal. They assume that 
trucks arrive in groups and that the information is revealed for the whole group at the same time. 
Using this assumption of rolling information, they use a myopic heuristic to get insights on the 
trade-off between information and optimality. Ku 10| consider a model where the trucks arrive 
in groups, and the retrieval order within in each group is unknown in advance. They propose a 
stochastic dynamic programming for this problem. They note that the size of decision tree grows 
exponentially with the bay size, making the computation time prohibitive. Thus they develop 
a heuristic called Expected Reshuffling Index (ERI) to approximately solve the problem. This 
heuristic, moves each blocking container to the column that minimizes the expected number of 
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future relocations. In our work, we model uncertainty as a 2-stage stochastic optimization problem, 
and we use a generalization of the A* algorithm combined with sampling and pruning techniques 
to find the near optimal solution of this stochastic optimization problem. 

This paper is structured as follows: Section 2 presents the A* Algorithm in the full information 
case. It describes the method, introduces lower and upper bounds and derives theoretical results for 
these bounds. Section 3 presents the asymptotic analysis of the CRP. Section 4 provides a stochastic 
scheme for the case of incomplete information and studies the value of information about container 
departure times. In Section 5, we present several experimental results for the average case analysis 
of the CRP in both the complete and incomplete information cases. Finally, Section 6 concludes 
and suggests future research topics. The proofs for all theoretical statements are given in the 
Appendix. 


2 Optimal solution for the CRP with full information 


In this section and for the sake of completeness, we present the A* algorithm applied to the CRP, 


as previously introduced by [2^. We also discuss bounds developed in the literature 
and derive new theoretical results on them. 


and 
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2.1 Description of the algorithm 

Before describing the A* algorithm, we review the Container Relocation Problem (CRP) and in¬ 
troduce a few notations used throughout the paper: We are given a bay B with C columns and P 
tiers, where C > P > 3. Initially N containers are stored in the bay. We label the containers based 
on their departure order, i.e., container I is the first one to be retrieved. The CRP corresponds to 
finding a sequence of moves to retrieve containers 1,2,... ,N (respecting the order) with a minimum 
number of relocations. For bay B, we denote the minimum number of relocation by Zopt{B). 

To find a solution, we use the A* algorithm, which is basically a move-based approach using a 
special decision tree. This approach takes 4 entries: B is the bay with N containers to be solved 
optimally. R{D) and S(D) are functions giving respectively upper and lower bounds on ZgptiD) for 
any configuration D. In this paper, upper bound functions that are considered also give us feasible 
solutions. Finally, N is the maximum number of nodes in the decision tree. 

The algorithm returns two results: The best incumbent (in objective function) found by A*, 
denoted z^, to which we associate the sequence of moves needed to find this incumbent called oa 
and the gap guaranteed between z^ and Zopt{B) denoted by y. Notice that if the algorithm did not 
reach N nodes, then the algorithm found an solution (i.e. y = 0 and z^ = Zopt{B)). 

Let us define the following notations: I is the level of the tree. A node is at level /, if I relocations 
have been done from the initial bay to get to this node. This is the reason why this method is called 
a move-based approach. Below we describe the algorithm in detail and introduce the notation used 
in the description of Algorithm [TJ 
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Algorithm 1 A* Algorithm 


1: procedure [zA,y]=A*{B,R,S,N) 

2: Pre-processing: 

3: while target container n is on top do 

4: retrieve target container n of B and n <— n + 1 

5: end while 

6: Initialize: 

7: Za oo, / <— 0, <— B, m <— 0 (number of nodes) 

8: while there exists a non-visited node do 

9: for all nodes B^ at level I do 

10: Mark node B^ as visited 

11: Lf(B') = R(B') + I and L(B') = S(B') + / 

12: Updating the incumbent: 

13: if Lf(B^) < z^ then 

14: Za ^ (i(B') 

15: end if 

16: Pruning: 

17: if U(B^) < L(B^) or L(B*) > Za then 

18: Prune node B* (i.e. stop branching from this node) 


20: Branching: 


for Every “Child” of B* do 

if m> N then 
Stop 

else 

add a non-visited “Child” to the tree at level I -I-1 and m = m 1 

end if 
end for 


end if 
end for 
/^ / + ! 
end while 


32: Gap: 


hmin = min (L(D)) 

D non-visited leaf 

y ^ Za ~ Imin 


35: end procedure 





The tree represents the sequence of relocation moves. Suppose n is the target container. If 
n is not blocked, it can be retrieved since a retrieval is optimal. In that case, no decision has to 
be made, therefore no new node is created. If n is blocked, the topmost blocking container needs 
to be relocated to another column. This column has to be chosen in order to minimize all future 
relocations. In order to do so, we generate all possible feasible moves for the blocking container by 
creating children of the corresponding node in a decision tree procedure. Considering all possible 
nodes gives the optimal solution. 

Building the whole tree of feasible moves would not be tractable since the number of nodes 
increases exponentially with the number of relocations. Consequently, we use certain decision rules 
to prune the tree with the guarantee of keeping the optimal path in the tree. 

Let be any configuration that appears at level I in the tree. By definition, 

S(B') < Zopf(B') < R(B') (1) 

Let us define the cumulative upper and lower bounds as the sum of the bound evaluated at the 
current state and the number of relocations already performed (i.e., / relocations have been done 
at level I of the tree) U{B^) = R{B^) + I and L(B^) = S(B^) +1 (line 11 of Algorithm [1]) . Notice that in 
this case 


L(B^) < ZoptiB) < U{B^), for all B^ in the A* tree (2) 

At any level /, the algorithm prunes paths using the following two rules (line 17 of Algorithm [1]) : 

(i) If L(B^) = LI(B^), then Zopt{B^) = LI(B^) and we can stop branching on this path. From this 
point, we can simply follow the feasible solution given by the upper bound. 

(ii) If L(B^) > Za (i.e. the lower bound L(B^) is larger than the best solution found so far Za) then 
this node can be pruned because the optimal solution for this node is going to be greater 
than its lower bound, hence greater than za- The best incumbent is updated if a B^ such that 
U(B^) < Za is found (lines 13, 14 of Algorithm [T]) . 

By construction, the above rules maintain the optimal path in the A* tree. 
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Figure 2: Bay with 3 tiers, 3 columns and 6 containers 
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U=5,L=4 


U=4,L=4 
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Figure 3: Example of A* algorithm using H as upper bound and So as lower bound (defined in 
subsection 12.2.111 

Example 1 Figure [3] presents the A* where the initial bay is shown in Figure [2j It uses upper 
and lower bounds that are presented in Section 12.21 None of the nodes at level 1 can be pruned. 
Now let us consider the 4 nodes at level 2 (the 4 rightmost configurations in Figure [3]). The third 
one uses rule (i) to stop branching from this node since lower and upper bounds are both equal to 
4. At this level = 4. Therefore we can also prune the first, second and fourth nodes using rule 
(ii). Therefore, the tree is complete at level 2. We follow the path until the third node at level 2, 
and then follow the feasible solution given by the upper bound. 

We know from Q] that the restricted CRP is NP hard. Therefore, there exist some instances for 
which the number of nodes needed to find the optimal solution with this method grows exponentially 
with the problem size. However to limit the computation time, we set a maximum number of nodes 
yV allowed in the A* tree. If the number of nodes reaches N, we return our best upper-bound as a 
feasible solution. In this case, we cannot insure optimality, but we can provide a guaranteed gap 
with the optimum solution. Let be the minimum of the cumulative lower bounds of every leaf 
of the tree that is not pruned by A*. Notice that ZgptiB) > Lmin- Thus z^ - ZoptiB) < Za - Lmin = 7 
(line 24 of Algorithm [1]) . 

We show later that given certain properties on the lower bounds, y is nonincreasing in N. In 
any case, there exists N* such that if Af > N* then y = 0. In our numerical average-case analysis 
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(Section [5]), we show how N and y are related. 


The size of the A* tree depends on the local lower and upper bounds used. The closer they 
are to the optimal solution at each node, the faster the tree is pruned, hence the faster it solves. 
In this paper, we define some of the possible candidates, explain these choices and provide details 
about the efficiency of the chosen bounds. Note that the algorithm can also be implemented using 
depth first search instead of breadth first search; in this paper, we implement the latter as we find 
experimentally that it is the most efficient one on average. 

2.2 The Candidate Bounds 
2.2.1 The Lower Bounds 

The counting lower bound. This bound was introduced by Kim and Hong in and it is based 
on the following simple observation. In the initial configuration, if a container is blocking, then it 
must be relocated at least once. Thus we count the number of blocking containers in B and we 
denote it Sq{B). Note that if a container blocks more than one container (for instance, container 6 
blocks containers 1 and 4 in Figure [2]), we count this container only once. 

Observation 2.1. For any configuration B, 

So(B) < Zopt(B) 

We define Lq as the cumulative bound computed with So- 
As mentioned in section El we want our algorithm to have the following property: As we increase 
Af, y should not increase. One way to enforce this is to insure that given any path of the tree, 
the cumulative lower bound is nondecreasing on this path. We will indeed show that Lq follows 
this property. Before proceeding let us define a few notations; Let C; be the column of the 
configuration and min(C;) be the index of the container with the smallest retrieval time in C;. Now 
suppose we want to move the blocking container r. If we relocate r to column Cj and r < min(C;), 
then we call such a move a “good” move and column c; is a “good” column for r. On the other 
hand, if r > min(c;), we call this move a “bad” move and c, a “bad” column for r. 

Proposition 2.2. For any configuration B, any level I > 0, any configuration B^ in level I of A* 
and any child of B^, we have Lo{B^'^^) > Lq{B^). 


The look-ahead lower bounds. Note that the counting lower bound (So) is only taking into 
account the initial configuration of the bay. By definition, a “bad” move implies a future relocation 
for the relocated container. Therefore we can construct lower bounds that anticipate “bad” moves 
and hence are closer to the optimal solution. This idea has been used by Zhu et al ( 2^) to define 
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the following family of lower bounds. For the sake of completeness, we redefine the bounds for¬ 
mally. In this paper, we prove in Proposition 12.51 that these lower bounds have the non-decreasing 
property as we branch in the A* tree. 

The basic idea is the following: We want to count unavoidable “bad” moves in any feasible 
sequence of moves. In order to do so, we need to keep track of the “maximum of minimums of 
each column of B” denoted by MM(B) = max (minici)). Suppose r is the first blocking container: 
a “bad” move for r is unavoidable if MM(B) < r. As we empty the bay, the maximum of minimums 
is changing. More precisely it depends on the sequence of moves that we use. We can, however, 
compute an upper bound on this value for every feasible sequence of moves. One way is to assume 
that every container that is relocated is discarded from the bay. At any time, the maximum of 
minimums in the “discarded” bay will be not smaller than if we computed it given any feasible 
sequence of moves. Before introducing formal notations, we give an example. 
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Figure 4: Example of discarded bays with 3 tiers, 3 columns and 6 initial containers 


Example 2 Consider the leftmost bay B in Eigure 01 Eirst, one can see that Sq{B) = 2 since 6 
is blocking 1, and 5 is blocking 2. Now in Bi, 6 is blocking 1 and MM{Bi) is 3; thus 6 has to be 
relocated at least twice. Therefore the optimum is at least 1 more than So- Now consider bay B 2 
where 1 and 6 have been discarded. 5 is blocking 2 and MM(B 2 ) is 4; thus 5 has to be relocated at 
least twice. Therefore the optimum is at least 2 more than Sq. 


Now we formalize this idea: Let ni be the smallest container in the bay. Let k be a container 
(k E [rii,... ,N}) and let ’Rk{B) be the set of containers blocking k and not blocking any container k' 
such that k' < k. 

Let B„^ = B and let B„^+i be the bay where container ni and containers in ’R„^{B) have all been 
discarded from B„^. By recursion, we can dehne a sequence of bays Bj^ for k E [ni + 1,... ,N}. 

Eor p E IN, we now dehne the p^^ look-ahead lower bound (denoted by Sp{B)): 

mm(j>+ni-l,N) 

Sp{B) = So(B) + L L Xir > MM(B,)l (3) 

k=ni 

where xi') is the indicator function. The corresponding cumulative lower bound is denoted by Lp{B). 
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Observation 2.3. For every eonfiguration B and p e 1,... ,N, 

Sp{B) < Zopt{B) 

Proposition 2.4. For p € 1,.. .,N and any bay B, we have Lp{B) > Lp_i(B). 

This result shows that as we increase p, Sp gets closer to the optimal solution. In Section [5l 
we numerically study the effect of the choice of lower bounds in the A* algorithm in terms of the 
number of nodes needed to solve a random set of instances. We observe that on average, most of 
the gain comes from the first look ahead. 

We remark that Sn(B) = S]\j-c{B) simply because when we reach N-C, there is always at least 
one empty column. Note that if k is the smallest integer such that Bj^ has an empty column, then 
the lower bound does not change, i.e., S]v(B) = Sjt-i(B). Thus, in practice, the process of computing 
the lower bounds terminates much before N — C. 

In the next proposition, we show that similar to Lq, the cumulative upper bound Lp has the 
desirable monotone property, i.e., they do not decrease as we branch in the A* tree. 

Proposition 2.5. For any p E configuration B, level 1 > 0, configuration B^ in level I and 

any child of B^ in the A* tree, we have Lp{B^'''^) > Lp{B^). 

In general it is hard to find the rate at which the lower bound increases on a given path. In 
Section [5l we show how the lower bound Sjv increases on the optimal path of random instances. 

2.2.2 The Upper Bound 

Any feasible solution can be used as an upper bound. Thus, we use heuristics to construct feasible 
solutions that serve as upper bounds. 

Ideally we would want a heuristic that is close to the optimal solution and at the same time 
easy to compute. Note that we need to compute an upper bound at every node. The heuristic 
proposed by Casserta et al. (Qj) seems to meet those criteria. For completeness, we redefine this 
heuristic. In this article we prove several properties of the upper bound that we will use in later 
parts (see Propositions 12.71 12.81 and 12.91) . 

The Heuristic H (fl]) Suppose n is the target container located in column c, and r is the 
topmost blocking container in c. We use the following rule to determine c*, the column where r 
should be relocated to. Recall that min(c,) is the minimum of column C;. We set min(c;) = N + 1 if 
Ci is empty. We have: 

{ argmin {min{c;) : min(c/) > r} if 3 c, such that min(C;) > r 

Ci6|l,...,C)\c 

argmax {min{c;)} otherwise 

06|1.Cl\c 
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The rule says: if there is a column where min(C;) is greater than r (r can do a “good” move), 
then choose such a column where min(c;) is minimized, since columns with larger minimums can 
be useful for larger blocking containers. If there is no column satisfying min(C;) > r (r can only 
do “bad” moves), then choose the column where min(c;) is maximized in order to delay the next 
unavoidable relocation of r as much as possible. We will refer to this heuristic as heuristic H and 
denote its number of relocations by Zh{B). 

Example 2 Consider the bay B in Figure [2j Using heuristic H, the sequence is going to be; 
relocation of 6 to column C3, retrieval of 1, relocation of 5 to column Ci, retrieval of 2, relocation of 
6 to column C 2 , retrieval of 3, relocation of 5 to column C 2 , retrievals of 4,5 and 6. Thus Zh{B) = 4. 
Notice that we had S 2 {B) = 4, so heuristic H is optimal for this initial configuration. 

By definition, H is a feasible solution, so it is an upper bound on the optimal solution. 

Observation 2.6. For any configuration B, we have Zopt{B) < Zh{B). 

The heuristic H has a certain number of useful properties stated below. 

Proposition 2.7. In a bay with C columns, for any configuration B with at most C containers, we 
have 


So{B) = Zopt{B) = Zh(B). 


( 4 ) 


Proposition 2.8. In a hay with C columns, for any configuration B with at most C + 1 containers, 
we have Si(B) = Zopt{B) = Zh{B). 

This result implies that if we use heuristic H as an upper bound and the Si lower bound in the 
A* approach, the tree will stop at most after N — C — 1 retrievals with the guarantee of an optimal 
solution. 

We provide two similar bounding results on H. 

Proposition 2.9. In a bay with C columns, for any configuration B with at most C + k containers, 
we have: 

• Zh{B) < Zopt(B) + 2, ifk = 2 
. Zh{B) < ZoptiB) + if3<k<C. 

Remark 2.10. The following example shows that this upper bound can increase as we branch in the 
A* tree. In Figure 0, bay (a) shows the initial configuration and bay (b) is one of its two “children”. 
It is easy to check that heuristic H needs 7 relocations for bay (a) and 8 for bay (b). 
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Figure 5: Heuristic H can increase on a path of A* tree 


3 An asymptotic analysis of CRP 

In this section, we study CRP for random large bays and we show that as the number of columns 
in the bay grows, the problem gets “easier” in the sense that the gap between the optimal solntion 
and our simplest lower bound (So), does not increase on average, and in fact it is bounded by a 
constant. The basic intnition is that, as the nnmber of colnmns grows, for any blocking container, 
we can find a “good” column with high probability. This implies that each blocking container is 
only relocated once with high probability. 

Throughout this section, we assume that initial configurations have P tiers, C columns, N con¬ 
tainers and that there are exactly h containers in each column, where h < P — 1. For more clarity 
we denote by Be such a bay with C columns.We assume that the initial confignration is a uniform 
sample among all possible such configurations. Notice that in that case, when C grows to infinity, 
N = hx C also grows to infinity. 

First let us explain how a uniformly random bay is generated. We view a bay as an array of 
P X C columns. The slots are numbered from bottom to top, and left to right from 1 to PC. For 
example, let P = 4 and C = 7, the second topmost slot of the third column is 11. The goal is to 
get a bay B with nniform probability, meaning each container is eqnally likely to be anywhere in 
the confignration, with the restriction that there are h containers per column. We first generate 
nniformly at random a permntation of {1,... ,N} called n. Then we assign a slot for each container 
with the following relation: B{j,i) = n{h X {i - 1) + j) for j < h and B{i,i) = 0 foi j > h + 1. One 
can see that each bay is generated with probability There is a one to one mapping between 
configurations with C columns and permutations of {1,.. ■ ,hC], denoted by She- Finally, we denote 
the expectation of random variable X over this uniform distribution by Ec[X]. 

Now we compute the expected counting lower bound. 

Proposition 3.1. Let So be the counting lower bound (defined in Section\^, we have 

Ec [So(Bc)] = och X. C, (5) 
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where is the expected number of blocking containers in one column and can be computed as 

h-l 

ah = J^kx ( 6 ) 

k=l 

and = P[i/iere are k blocking containers in a column with h containers] can be computed by re- 
crusion as 

1 1 
Sh > 0, po,?/ = ^ and Vfc > 1, p^^ = -p^-j+xh-j 

;=i 

Remark 3.2. Note that ah only depends on the distribution of the relative order of the h containers 
in one column (and not on the actual labels). Thus the expected number of blocking containers in 
one column only depends on its height. 


The major result of this part is the following. In the asymptotic case where the number of 
columns increases to infinity, the expected optimal number of relocations is asymptotically propor¬ 
tional to the expected number of blocking containers (the counting lower bound). 


Theorem 3.3. Let So be the counting lower bound (defined in Section\^ and Zgpt be the optimal 
number of relocations. Then for C >h + 1, we have 


Ec [zopK^c)] 

1 <- — - - < f(C) 

- Ec[So(Bc)] ’ 


( 7 ) 


where 


m 


-ic: 


( 8 ) 


where K is a constant defined by equation m- 


The proof of Theorem 13.31 is given in the Appendix. Here we just give an intuition of the 
proof. We show that as C grows, with high probability ^c+ii^opti^C+i)] “ Ec [zopt(Bc)] is exactly 
ah. Therefore, for C large enough, Ec [zopt(Bc)] essentially behaves like ah X C, which is equal to 
Ec[So(Bc)] (according to Proposition 13.11) . 

In the next corollary, we show that the optimal solution of the unrestricted CRP has a similar 
asymptotic behavior. We remind that the unrestricted CRP refers to the problem where we can 
also relocate non-blocking containers. 


Corollary 3.4. Let Zgeni^c) be the optimal number of relocations for the unrestricted CRP. For 
C >h + 1, we have 


1 < 


Ec [^^^(Bc)] 


a/jC 


</(C) 


where f is the function defined in Theorem |g.3l 


(9) 


15 




The above theorem gives insights on how the expected optimal solution of the CRP behaves 
asymptotically on random bays. To conclude this section, we show experimentally that the same 
result holds for heuristic H, i.e., the ratio of E [zh(Bc)] and E [So(Bc)] converges to 1 as C goes to 
infinity. For each size C, we compute both expectations over a million instances, take their ratio 
and plot the result in Figure [6al Notice that we have 1 < ^ e[s^(Bc)] ’ Figure [6a] also 

shows experimentally that Theorem 13.31 holds. 




(a) Convergence of the ratio (^l Convergence of the difference E [zh(Bc)] - 

"" E[So(Bc)] 


Figure 6: Simulations for the Asymptotic Theorem 


First, note that Figure [6a| implies that the relative gap between heuristic H and So shrinks to 
0 as C increases. Moreover we have ^ and thus the relative gap 

of H with optimality also converges to 0 as C grows to infinity. 

In the proof of Theorem 13.31 we also study the function g{C) = E^ [zopf{Bc)] “ [So(Bc)]- 

Note that ^(C) < Ec [zh(Bc)] “ Ec [So(Bc)] where the right-hand side of the inequality is the 
function plotted in Figure [60 The plot shows that g(C) < 1.25 for all C, meaning that g(C) is 
bounded as we proved in Theorem 13.31 Moreover, the plot implies that heuristic H is on average 
at most 1.25 away from the optimal solution, so heuristic H is relatively more efficient in the case 
of large bays. Intuitively, the probability of having a good column converges to 1, as we increase 
the number of columns; hence the problem tends to become easier as C grows. 

Finally, in the proof, we note that the rate of convergence for the optimal solution to So is at 
least Interestingly, we can infer from Figure [60 that the rate of convergence of the ratio for 
heuristic H is also proportional to 


4 CRP with Incomplete Information 

The A* algorithm as explained in Section [21 relies on the assumption that the departure order of 
containers is known in advance. However, in practice, we often only know the departure order of 
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some of the containers (those that are going to depart in the near future). 

In this section, we explain how the A* algorithm can be adapted in a Dynamic Programming 
(DP) framework for solving the CRP in the case that some of the decisions should be made with 
incomplete information. We use the A* algorithm to obtain an approximate solution for this 
problem. In what follows, we describe the setting of the problem, introduce some notations, and 
explain the algorithm for the CRP with incomplete information. Moreover, we show how to bound 
the approximation error. In Section [Sj we present the results of computational experiments. 

The CRP with incomplete information involves retrieving N containers from a bay with C 
columns and P tiers, where partial information about the departure order of the containers is 
initially available. To model the CRP with incomplete information, we discretize time into time 
steps of equal length and assume that each move (a relocation or a retrieval) takes exactly one 
time step. Further we assume that the label (or index) of a container indicates the earliest time 
that it can be retrieved (i.e., container n can be retrieved at time n or later). Usually, very little 
information is available about the containers that are going to be retrieved far in the future. Thus, 
it is reasonable to assume that at any given time step f, we only know the departure order of a 
certain number of containers in the bay (the containers that are going to be retrieved within a 
short time horizon after t). We refer to such containers in the bay as known containers. Similarly, 
we refer to the remaining containers in the bay as unknown containers. By definition, all unknown 
containers have larger indices (i.e., later departure times) than the known containers. 

As time passes, some of the known containers are retrieved and as more information becomes 
available, some of the unknown containers become known. In general, the information might be 
updated multiple times during the retrieval process (every time that the arrival time of a truck is 
provided by the truck driver, the information gets updated and some unknown containers become 
known). In the most general case, the information can be updated every time step. Alternatively, 
we can consolidate several small pieces of information into one or a few pieces, and assume that 
the information is revealed at P different times during the retrieval process (i.e., P is the number 
of time that information is updated). In this case, we have a multi-stage problem and P sets of 
decisions need to be made. 

In this paper, we focus on a 2-stage setting; we assume that a subset of containers is initially 
known and that the departure order of all the remaining ones becomes known at time t*. We denote 
the set of known containers by K and assume that containers {1,2,..., |K|} are those known at time 
zero. Similarly, the set of unknown containers is denoted by U, and containers {|K| + 1,...,N} 
become known at time step t* > 0. We refer to this setting as the 2-stage CRP since there are two 
types of decisions that need to be made: first-stage decisions (retrievals and relocations before time 
step f) and second-stage decisions (retrievals and relocations after time step f). We assume that 
before P, we have probabilistic information about the containers in U, meaning that we know the 
probability of realization of each possible departure order. Such information can be obtained from 
historical data or from an appointment system that provides some estimate of departure times of 
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the containers (for example a time window for retrieving each container). We denote the set of 
possible departure orders of the containers in U (possible scenarios) by From now on, we assume 


1 

that all scenarios are equally likely, i.e., the probability of each scenario is —, and the number of 

l«l 

scenarios is (N — |K|)!. Note, however, that we could use the algorithm to solve the CRP with any 
other probability distribution on the departure order of unknown containers. 

We use a 2-stage stochastic optimization technique to solve this problem, where in the first-stage 
we minimize lE[z] as follows: 


min E[z] = Y ^z{B{q)). (10) 

qeQ 

where B{ci) is the resulting bay when scenario q is realized, z(B(q)) is the total number of 
relocations for B(q), and ai,..., at’-i are the hrst-stage decisions. The 2-stage problem can be 
solved with the A* algorithm as follows: 

( 1 ) . We build the tree with t* — 1 moves for time-steps 1,2,...,r — 1, in a similar way as 
illustrated in Figure [3l we denote this tree by 

(2) . For each node at level t* — 1, we need to compute the expected number of remaining 
relocations. We enumerate all possible scenarios and solve the CRP with complete information 
corresponding to each scenario, using the A* algorithm. 

(3) . We find p* (the optimal path or sequence of moves) that minimizes the expected total 
number of relocations over all paths. 

(4) . Once p* up to time-step P — 1 is selected and we observe the information at f, we use the 
A* algorithm to solve a specific instance through the end. 

Notice that to find the optimal path for time interval [1, P — 1], we need to solve up to (C — 
l)(f-i)(]V - |K|)! instances in Step (2) with the A* algorithm. Although A* is fast, the number of 
scenarios is prohibitively large and enumerating all scenarios is not feasible due to limited resources 
of memory and long computation time. We next explain how we use sampling and pruning to 
overcome these issues. We also quantify the error incurred as a result of sampling and pruning. We 
refer to the resulting algorithm as ASA* (Approximate Stochastic A*). 

Limiting the number of possible scenarios on each path. We overcome this issue by 
sampling S scenarios on each path and computing the number of relocations for the sampled 
departure orders rather than for all possible orders. For each path p, let z be the number of 
relocations averaged over samples. Also let E[z] be the true mean. To determine the number of 
samples needed to get a good approximation, we use the following version of Hoeffding’s inequality: 

P(|E[z] -z| > 6) < 2exp( ), (11) 

\Vmax ^min) j 
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where 6 is a pre-specified level of desired precision, and rmaxl'^min are lower/upper bounds on 
random variable z. 

Note that we can set r^in = 0; for we do not have a tight bound, but we can use N{P - 1) 
as an upper bound (since each of the N containers is blocked by at most P - 1 containers). Let us 
denote the desired probability for bounding the error (i.e., RHS of (fTT]l l by e. For a given 6 and e, 
the required number of samples can be computed as follows. 


S > 


1*2 In — 

' max ^ 2 
-262 


( 12 ) 


By sampling from the possible scenarios on each path, we can significantly reduce the number 
of scenarios (and thus the computation time). For example, for a bay with 7 columns, 4 tiers, 
21 containers, and |K| = 6, the total number of possible scenarios on any of the paths at time t* 
is about 10^^. Using inequality (fl^ . the total number of scenarios would be around 30,000 for 
6 = 0.5, e = 0.05, and r^ax = 63. Note that by sampling, we incur an error and ASA* may choose 
a suboptimal path Pasa where E[Zp^g^] > E[Zp*] (recall that p* is the optimal path that would 
be chosen without sampling). In the next proposition, we show that such an error, denoted by 
ei = E[Zp^g^] - E[Zp*], is bounded in expectation (over a uniform distribution on the initial bay). 


Proposition 4.1. Suppose for each path, we estimate the number of relocations using S independent 
samples, where S is given in m- Also suppose ASA* chooses path Pasa as the optimal path, and 
ei = E[Zp^g^] - E[Zp-]. We have E[ei] < 26 / ^ ^ • 

Pruning the paths of To address this issue, we use the upper bound and lower 

bounds to prune the nodes of similar to the A* algorithm. However, since some of the 

containers are unknown before t*, we have to compute the expectations (E[L] and E[L1]). Again, 
we use the idea of sampling and we estimate these values by computing L and U using S samples, 
where S is obtained from Inequality (jl2jl . Because of the sampling error, we may prune an optimal 
path by mistake, resulting in an error that is illustrated in Figure [71 This error is the difference 
between E(Llp) (the true upper bound of the path with the minimum estimated upper bound), and 
E(Lp) (the true lower bound of the path selected by ASA* for pruning). Intuitively, this is because 
E(Llp) is the worst we would achieve if we prune p (by mistake), and E(Lp) is the best we could 
achieve if we do not prune p. 

In the next two propositions, we show that the error due to pruning is bounded, whether we 
prune some paths once at t* — 1 (resulting in error 62 ) or prune some paths at several time-steps 
before t* — 1 (resulting in error 63 ). 


Proposition 4.2. Suppose that for each path p at time-step t* — 1, we estimate the expected lower 
and upper bounds (Lp and Up) from S samples, where S is given in (fTTIl . Also, suppose ASA* 
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Figure 7: Illustration of the error due to pruning: the thick line segment is the error as a result of 
pruning the optimal path by mistake. 



chooses to prune one or more paths. We have E[e 2 ] < 26 


Proposition 4.3. Suppose we prune some paths at ni time-steps h/ti, ■■■rim < t* — !■ At each 
time of pruning and for each path p, we estimate the expected lower and upper hounds (Lp and Up) 
from S samples where S is given in Also, suppose that at each time, i, of pruning, ASA* 

chooses to prune one or more paths (denoted by pi). The expected total loss, E[e 3 ], is bounded 


by m 


€ , mm 


^2^ 


+ 


-ln{^)n ^ fi. 

' 9 ' £ mm 

—— {-W 
2 


TL -—max 




where Up 


max{Up^,. ..,U, 


pm > ^ 


di — Lp. Up., djyiin — minjrfi,..., 4^?}. 


In Propositions 14.1114.31 we bound the loss that can be incurred by pruning some paths at time 
t* — 1 (ci and 62 ) or at t < P — 1 ( 63 ). Notice that for fixed 6 and 6 , the errors ei and 62 are 
independent of the bay size if we increase the number of samples S according to (jl 2 l] : thus if we 
prune paths only at t* — 1, the loss remains unchanged and the relative loss (as a percentage of 
total relocations) decreases as the bay gets larger. Therefore, for large bays and “hard-to-solve” 
configurations, one can pick larger 6 and e that result in a smaller number of samples, and the 
relative error would still be small. 

Table [T] shows ei , €2 and £3 . The losses are very small even for combinations of 6 and e that 
result in reasonably small number of samples. For example, for 6 < 1 and e < 0.1, E[ei] and E[e 2 ] 
are no more than 2. Because and 62 are independent of the bay size, the relative error ( ^ ) 

decrease as the bay gets larger (E[z] is the average number of relocations when full information is 
available). For the 63 , this measure is almost constant for different bay sizes and can be controlled 
by changing e, 6 , and m (number of times that we do pruning at t < t* — 1), 
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5 

e 

E[ei] and E[e 2 ] 

0.1 

0.01 

0.15 

0.1 

0.05 

0.18 

0.1 

0.1 

0.2 

0.5 

0.01 

0.77 

0.5 

0.05 

0.92 

0.5 

0.1 

1.02 

1 

0.01 

1.54 

1 

0.05 

1.85 

1 

0.1 

2.05 


C 

Hes] 

% error ( 

10 

0.91 

0.07 

15 

1.36 

0.073 

20 

1.81 

0.074 

25 

2.26 

0.075 

30 

2.72 

0.076 

35 

3.17 

0.076 

40 

3.62 

0.076 

45 

4.07 

0.076 

50 

4.52 

0.076 


Table 1: Left table: Expected loss due to sampling and pruning at f* — 1, for C = 7, P = 4, and 
N = 21; Right table: Expected loss due to pruning at f < P - 1 for e = 0.05, 6 = 0.5, dmin = I, 

—max 

Up = 2N, and m = 5 

In Section [5l we use ASA* to solve the CRP with incomplete information for a bay with 7 
columns and 4 tiers and with different amounts of information initially available. Moreover, we 
introduce a heuristic for this problem and compare the results of ASA* with those of the heuristic. 


5 Experimental Results 


In this section, several experimental results are presented to further understand the effectiveness 
of our algorithms in both complete and incomplete information settings. 

In the complete information case, we study thoroughly the A* algorithm and the effect of 
parameters on the efficiency and performance of the algorithm. First we show that using the 
look-ahead lower bound improves dramatically the computational tractability of the algorithm. 
Moreover, our experiments shows that on the optimal path of the tree, the look-ahead lower 
bound reaches the optimum solution after a few levels. We also show the trade-off between N 
(maximum number of nodes in the tree) and the number of instances solved optimally. Further we 
introduce a new class of heuristics, the Tree Heuristic (TH-L), compare it with 3 existing heuristics 
(01) 0) Ql) and show that it outperforms the existing heuristics. 

In the incomplete information setting, we show through experiments that the ASA* algorithm 
is fast and efficient and most of the instances are solvable within a reasonable time for medium¬ 
sized instances. We introduce a myopic heuristic which expands on the H heuristic in 0. Using 
the ASA* algorithm and the heuristic, we study the value of information. Our simulations results 
show that while more information results in fewer relocations, the marginal value of information 
decreases with the level of information. We also use the myopic heuristic to examine the effect of 
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the level of information on the number of relocations for different bay sizes. We show that for any 
level of information, the ratio of relocations with incomplete information and complete information 
converges to a constant as the bay gets larger. 

5.1 Experimental results for complete information 

In most of the experiments, we consider medium-sized instances of the CRP. We consider bays of 
size 4 tiers and 7 columns with 21 containers (P = 4 = h + l, C = 7 and N = 21 = hC with 3 
containers per column). We generate randomly 100,000 instances using the uniform distribution 
on bays described in Section 4. 

The effect of lower bounds on the size of the A* Tree. In Section [21 we introduced a 
family of lower bounds (So, Si,... ,S]v). Clearly Sn is more computationally expensive to use than 
So, but we proved that Sjv was a tighter lower bound. We show here that using Sjv improves the 
performance of the A* algorithm significantly. 

For each instance, we solve it using 4 types of lower bounds So, Si, S 2 and S 21 (here N = 21). 
For each of them we record the number of nodes needed to solve the instance optimally. Figure [8] 
presents the results. First, we show in the box-plots, the distribution of the number of nodes for 
each of those lower bounds. Second, we give the average number of nodes. We give two main 
insights from this experiment. First, the average number of nodes needed to solve with S 21 is 1/3 
of the number of nodes with Sq. Also, we point out that using Si and S 2 instead of So decreases 
the average number of nodes by 1/3 and 1/2, respectively. The fact that the number of nodes is 
decreasing as we use tighter lower bounds is not surprising and it is aligned with what we proved 
in Section |2J The more surprising insight from the experiment is that introducing lower bounds 
that look just one or two steps ahead (i.e.. Si and S 2 ) makes a great improvement in terms of 
performance of the A* approach, without affecting the computation time significantly. 

Second, it can be seen that the box is concentrated around 0 for each lower bound, which implies 
that most instances are solved within a few hundred nodes. What influences the average number 
of nodes are the “hard-to-solve” cases where a large number of nodes is needed. Using S 21 instead 
of So makes the A* algorithm more efficient by decreasing the number of nodes by a factor of 3 for 
those cases. From now on, we use S 21 as the lower bound. Recall that actually we only need to 
compute S]( loi k < N - C, as Sjt+i,. ■■ ,Sn are all equal to S^. Taking advantage of this property 
further improves computational tractability. 

The convergence rate of the lower bonnd. After introducing the lower bounds, we have 
shown in Propositions 12.2) and 12.51 that on any given path, the lower bound is non-decreasing. The 
efficiency of the algorithm mainly depends on the rate of increase of the lower bound on each path. 
If this rate is high, then the algorithm has a greater chance to find the optimal solution with fewer 
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Figure 8: The effect of the Lower Bound on A* 

nodes, as the tree is pruned faster. The lower bounds can either “collapse” with the upper bounds, 
or exceeds the best incumbent available. 

In this experiment and for the sake of clarity, we only focus on the rate of convergence of the 
lower bound on the optimal path. For each instance, we find the optimal path, and at each level /, 
we compute the difference between and Zopt{B^)- We average this difference on all instances 

and plot the results in Figure [9] 



level of the A* Tree 

Figure 9: The convergence rate of the lower bound on the optimal path 

First of all, we can see that even at level 0 (meaning at the root node), the average difference 
is less than 0.25. This shows that on average the lower bound is close to optimal, therefore it does 
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not need to increase a lot to reach the optimal solution. Second, we can see that after level 10, the 
average gap is less than 0.001, meaning that after 10 relocations, in most cases, the lower bound 
has reached optimality on the optimal path. Therefore, this path will terminate at level 10 if the 
upper bound also collapsed to the optimal. Finally, the trend of the curve shows that the gap 
between optimal and the lower bound decreases faster at the beginning of the tree. A simple way 
to explain this is that the lower bound increases when a future “bad” move can be predicted. But 
such bad moves become harder to predict as the bay gets emptier (there are more empty columns 
in the bay). 

The effect of N. By construction, the A* method is tunable. In this experiment, we study 
the impact of N on the average guaranteed gap with optimality, and the percentage of instances 
solved optimally. Notice that from Propositions 12.21 and 12.51 we know that the guaranteed gap is 
a non-increasing function of N. We solve each instance with different N and we record the gap 
with the optimal solution obtained by the A* algorithm. We show the results in Figure [TOl On the 
left, we give the average gap as a function of N and on the right, we show the percent of instances 
solved optimally. 




(a) Average Gap as a function of N (b) Percentage of instances solved optimally as 

a function of N 

Figure 10: The Effect of N 

First, we observe that the average guaranteed gap at the root node is very small (less than 0.4). 
Also, in more than 70% of instances, the optimal solution is found at the root node. Second, as we 
increase N, the gap sharply decreases and the number of instances solved to optimality increases. 

This experiment is very useful from a practical a point of view in order to determine N. We 
can set an average gap that we allow in our solutions and infer the minimum N that is needed to 
achieve this performance on average. For example for bays of 4 by 7, if we allow for a gap of 0.05 
on average, we can set our N to 100. 

This experiment is related to the first experiment. As shown in Figure [HI most instances only 
require few hundreds of nodes, but there are some that require hundreds of thousands. This explains 
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the “concavity” of the function in the left plot and the “convexity” of the function in the right 
plot. In conclusion, this experiment highlights the trade-off between the quality of the solution 
(represented by the average guaranteed gap) and its tractability (represented by N). 

Benchmark of a new heuristic and existing heuristics. Many heuristics have been devel¬ 
oped for the CRP. From a practical point of view, heuristics are valuable since they are fast and 
easy to implement in container terminals. Therefore, it is relevant to evaluate their performance 
to advise operators. In order to benchmark existing heuristics, one can measure: 

• The distribution of the difference between heuristics and the optimal solution; and 

• The expected performance ratio of the heuristic dehned as 

, '^heuristic ~ ^opt , , 

PR{heuristic) = -, (13) 

Zopt 

where Zii^uristk is the number of relocations in the heuristic solution. PR shows the relative 
gap with optimality for a given heuristic. 

From Figure [H if we set N to 400,000, then all 100,000 instances are solved optimally using the 
A* algorithm. In addition to heuristic H, we study two other existing heuristics presented by Kim 
and Hong (KH from which uses the estimation of future relocation) and Petering and Hussein 
(LA-5 from (l^ that takes into account “repositioning moves”, i.e. repositioning containers that 
are not in the same column as the target container). 


Gap with 

Optimal 

Heuristic H 

Distribution 

Heuristic KH 

Distribution 

Heuristic LA-5 

Distribution 

Heuristic TH-2 

Distribution 

0 

87.0% 

31.8% 

83.5% 

95.7% 

1 

11.4% 

21.4% 

13.7% 

3.98% 

2 

1.4% 

21.3% 

2.0% 

0.3% 

> 3 

0.2% 

11.4% 

0.3% 

0.02% 

B[PR] 

1.44% 

16.0% 

1.81% 

0.44% 


Table 2: Benchmarks of Heuristics on 100,000 instances 


We use the same idea as in the A* algorithm and introduce a new class of heuristics that can 
improve any existing heuristics; we refer to this class as Tree Heuristic (TH-L). The basic idea of 
TH-L is to take the L best columns and branch on them to construct a decision tree. The L best 
columns can be chosen using any of the existing heuristics that compute a score for each column. 
Using this principle and considering several good candidates for each relocation, we are less likely 


25 







to make a mistake. Here, we implement the TH-L with the H heuristic. The algorithm is presented 
in Algorithm [2j In our experiment, we set L = 2. Note that TH results in less or the same number 
of relocations compared to H since the path of H is included in the tree of TH-L. 

Algorithm 2 Tree Heuristic 
1: procedure [Zjh] = TreeHeuristic{B,L) 

2 : Zju <— 0 

3: while B is not empty do 

4: Retrieval: 

5: if target container n is on top of its column then 

6: Retrieve n form B 

7: Relocation: 

8 ; else 

9: r <— topmost blocking container 

10: Cl «— sort arg increasingly {min(Q)| min(c/) > r} 

oell .Cl\c 

11; C 2 sort arg decreasingly {min(cOI min(cO < r} 

06|1.Cl\c 

12: C3<— [Ci,C2] 

13: S ^ C3[l : L] 

14: ZjH = ZjH + min{TreeHeuristic (B where r moves to column s,L)} 

sgS 

15: end if 

16: end while 

17; end procedure 


Results for the four heuristics are summarized in Table [2l First, H is optimal in most instances 
(87%). This is one of the main reasons that heuristic H was chosen as an upper bound in the A* 
method. Second, TH-2 is indeed closer to optimality than H, KH and LA-5, in distribution and in 
terms of the average performance ratio. 

Finally, notice that the percentages for LA-5 do not sum to 100% since there are cases for which 
LA-5 is better than A*, which is because LA-5 considers “repositioning moves”, i.e., it solves for 
the unrestricted CRP. As a result, for 0.5% of the 100,000 instances, LA-5 solves the instance with 
one relocation less than A*. 

In our last experiment for complete information, we study how the parameter L affects the 
performance of the TH-L heuristic. We consider the same 100,000 instances and solve them with 
L varying from 1 to 6, and we record their performance ratio. Notice that TH with L=6 considers 
all possibilities for the blocking container therefore it gives the same solution as the A* without the 
use of bounds. Thus PR(TH-6) = 0. Further note that L = 1 gives heuristic H. 

The main observation is that the marginal gain of branching is maximum when we increase L 
from 1 to 2. By considering two promising columns instead of one, TH-2 finds better solutions 
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L 

1 

2 

3 

4 

5 

6 

PR (TH-L) 

1.44% 

0.44% 

0.27% 

0.20% 

0.16% 

0% 


Table 3: Effect of parameter L on the performance of heuristic TH-L 


for most instances where H was not optimal. Note that increasing L (considering more candi¬ 
date columns) will further improve the solution; however the gain from more branching is small 
considering the exponentially increasing cost of computation. 

5.2 Experimental results for the CRP with Incomplete Information 

Number of nodes and computation time for ASA* . Stochastic optimization methods that 
are based on enumerating scenarios are usually computationally expensive. The ASA* algorithm, 
however, is fast and tractable due to the use of sampling and pruning that allow for suppressing 
many nodes in the tree. More importantly, the ASA* is tunable in the sense that one can set e and 
6 to change the number of nodes and thereby solve an instance within a desired amount of time, 
and yet ensure that the loss from using large e or small 6 is bounded (as shown in propositions 

Msm- 

Figure lllal shows the cumulative distribution of the number of nodes (after pruning) for the 
tree up to r=ll, for a bay with 7 columns, 4 tiers, 3 containers per column, and |K| = 11 (Recall 
that t* is the time step at which the set of unknown containers become known and |K| is the number 
of containers that are initially known). We implement ASA* with 6 = 0.5 and e = 0.05. It can be 
seen that for half of the instances, the tree up to t* has 100 or fewer nodes. Also, about 90% of 
instances have 200 or fewer nodes. Note that the nodes of the trees that are constructed after time 
t* for solving instances corresponding to different scenarios do not have too much of an effect on 
computation time because the A* algorithm is very fast. Figure fllbl shows the cumulative density 
of the average computation time. It can be seen that half of the instances are solved in less than 
two minutes and 90% of the instances are solved in 15 minutes or less. 

Although the ASA* algorithm is fast for medium-sized bays, it might not be tractable for large 
bays. We next introduce a myopic heuristic and use both ASA* and the heuristic to provide several 
insights on the value of information. 

Myopic Heuristic. This is an extension of the heuristic H that we explained in Section [2j 
Similar to the setting that we explained in Section 01 the problem is to retrieve N containers from 
a bay with C column and P tiers. In the incomplete information setting, the departure order of 
containers {1,2,..., |K|} are known at time zero; the departure order of containers {|K|, |K| -I-1,..., N} 
become known at time step t* > 0. 

Suppose n is the target container located in column c. If n is not blocked by any container, it 
is retrieved without any relocation. Otherwise, let r be the topmost blocking container in column 
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Total number of nodes after prunning 
(a) ASA' implemented with d = 0.5 and e = 0.05 



(b) Experiments were done on desktop computer with 
Intel i73770s processor (3.90 GHz) and 16Gb RAM. 


Figure 11: Tractability of the ASA* algorithm 


c, and c* be the column where r should be relocated. The myopic heuristic determines c* using the 
same rules as the H heuristic explained in Section [2l except that we assign an index of N + 1 to all 
unknown containers and we set min{ci) = N + 2 if c; is empty. Ties are broken arbitrarily. In the 
following experiments we study myopic heuristic and compare its performance to that of ASA*. 

Value of information. To study the effect of the level of information initially available (|K|), we 
take 1000 random instances of a bay with 7 columns, 4 tiers, N = 21 containers, and 3 containers 
per column. We solve each instance with 6 levels of information: |K| = r0.25N], r0.375N], [O.hN], 
r0.625N], r0.75N], and ["O.ON], using the ASA* algorithm and the myopic heuristic (["x] is the 
smallest integer larger than x). For all cases, we hx t* at r0.25N] + 1 to ensure that at every time- 
step before t*, at least one container is known. For each of the 12 cases, we compute the average 
of relocations over the 1000 instances. We then compare the relative gap of each case with the 
average relocation for the CRP with full information: and , where Zmh is 

the number of relocations obtained by the myopic heuristic. 

Figure I12al shows the relative gap in relocations for different levels of information. With the 
ASA* algorithm (implemented with 6 = 0.5 and e = 0.05), the gap is about 8% when 25% of the 
containers (6 containers) are known at time zero. The gap reduces to 3% when half of the containers 
are initially known and is almost zero when 90% of containers are known at time zero. The same 
behaviour can be observed for the heuristic. In both cases, the marginal value of information 
becomes smaller when more information is available. This is more significant for the heuristic. For 
example, when the level of information increases from 25% to 50%, there is a significant drop in 
the gap; then the gap decreases more slowly and approaches zero at 100% information. Note that 
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with 100% information, the myopic heuristic is the same as the H heuristic. 

To get an insight into the value of information for the myopic heuristic, recall that it behaves 
similar to the H heuristic as long as min{ci) for all columns are known. Since all unknown containers 
have larger indices than the known containers, knowing at least one container in each column is 
sufficient to obtain the same mm(c/) as in the heuristic H. Thus, after some point, having more 
information does not have much of effect on the number of relocations when we use the myopic 
heuristic. 

Figure I12bl shows how the gap between the myopic heuristic and ASA* shrinks as the level of 
information increases. When 25% of the containers are initially known, using the myopic heuristic 
results in 12% more relocations on average (compared to ASA*). This gap drops to less than 2% 
when all containers are known. Note that when more than 50% of the containers are initially 
known, the solution provided by the myopic heuristic is reasonably close to the solution of ASA* 
(less than 5%). Therefore, the myopic heuristic can be used in practice as it is easy to implement 
and efficient. 
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rithm: relative gap in relocations for different levels of 
information 


Figure 12: Experiment results for 7x4 bay 


It is interesting to see the effect of different levels of information on the number of relocations in 
larger bays. Figure [13] shows the average relative gap in relocations for bays of different sizes. For 
each bay size, the average relative gap is computed for 100,000 instances, using the myopic heuristic. 
The most important observation is that the ratio of E[za 4 h] and E[zh] does not constantly increase 
with bay size. And, for each information level, this ratio converges to a constant as the bay gets 
larger. Two other observations can be made from Figure [T3| First, the converging ratio for large 
bays drops fast as the level of information increases. For example, when 25% of containers are 
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initially known, the ratio converges to 1.4. This number drops to 1.03 when the level of information 
is 75%. Second, the rate of convergence is much faster when more information is available. For 
example, when 75% or more of the containers are initially known, the ratio is almost constant 
independent of the bay size. When 25% of the containers are known, the ratio converges to 1.4 for 
bays with 40 or more columns. Last, note that the asymptotic behaviour of the myopic heuristic 
in Figure [131 is as an upper bound on the performance of the ASA* algorithm. 

What is shown in Figure [l3] provides useful insights for port operators. Considering the gain 
(fewer relocations) at each level of information, port operators can design appointment systems 
that capture this gain by offering faster or discounted service to the customers who provide their 
arrival time in advance. 



Figure 13: Asymptotic behaviour of myopic heuristic: approaches a constant as the bay gets 

larger. 


Comparing the myopic heuristic with existing heuristic. To the best of our knowledge, 
only a few recent papers studied CRP with incomplete information, and developed heuristic algo¬ 


rithms for this problem [lOl . l2ll |. Here we compare our myopic heuristic with the RDH (revised 
difference heuristic of [^ 1 and show that myopic outperforms the RDH. Zhao et al. 21] show the 
percentage savings of RDH over a nearest relocation strategy for different bay sizes and for different 
amounts of information initially available. The greatest savings is about 50% and is realized for a 
bay with 12 columns and 7 maximum height, with 50 or more containers initially known (see Figure 
13 of 


2l[|L The total number of containers is not indicated for this simulation, so we assume the 


bay is full with 72 containers. (With 50 known containers, the level of information is about 70%). 
For a bay with 12 columns and 3 containers per column, the maximum savings is about 40% and 
is realized when 22 or more containers are known (i.e., when the information level is about 60%). 
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Figure 14: Percentage of saving of myopic heuristic over nearest relocation strategy. 

In Figure we show the percentage savings of the myopic heuristic over the nearest location 
strategy for bays with different numbers of columns, with 4 tiers and 3 containers per column. 
The savings are shown for 4 different levels of information. It can be seen that when level of 
information is 50%, the saving is about 48%, which is greater than the saving for RDH when even 
more information (about 70%) is available. When the level of information is 25%, the savings of 
the myopic heuristic is about 35%, whereas for the same setting, RDH results in less than 25% 
savings over the nearest relocation strategy. 

6 Conclusion 

Managing the relocation moves is one of the main challenges in the storage yard of container 
terminals and has a direct effect on the costs and efficiency of yard operations. The Container 
Relocation Problem (CRP) is notorious for its computational intractability and most research 
studies have designed heuristics in order to solve the problem, particularly for large bays. 

In this paper, we revisited a fast and efficient method, the A* algorithm that provides optimal 
solution for the CRP. We proved several properties of the method as applied to the CRP and 
assessed its performance and computational tractability through experiments. We also studied the 
asymptotic behavior of optimal solutions of the CRP and showed that the ratio of the optimal 
number of relocations and a simple counting lower bound converges to I as the bay gets very 
large. This gives strong evidence that CRP is easier in large bays. Moreover, we showed through 
experiments that the H heuristic (introduced in Q]) has the same behavior. An important result 
of our experiments is that it is recommended to apply the heuristic H to large blocks of containers 
(several bays) in order to minimize the loss from using a heuristic rather than optimization methods 
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(that are computationally expensive). We also presented an improvement over H heuristic, and show 
through experiments that it outperforms existing heuristics. 

Further, we extended our study to the CRP with incomplete information, which is an important 
case in practice because the retrieval order of containers are usually not known far in advance. 
We introduced a two-stage approximate stochastic optimization framework (ASA*) that optimizes 
the expected number of relocations given a probabilistic distribution on the departure order of 
containers. We also presented a myopic heuristic that is very fast and efficient for the CRP with 
incomplete information. We used ASA* and the heuristic to study the value of information for 
different bay sizes and showed that the relative gap between ASA* and the heuristic shrinks as 
the level of information increases. In fact, when the available information is more than 50%, the 
myopic heuristic is very efficient and can be used in practice. We also showed that the ratio of the 
number of relocations between heuristic H and the myopic heuristic converges to a constant fast 
as bays get larger, which again implies that the loss from using the myopic heuristic is minimized 
when applied to large bays. 

Our paper opens directions to many further studies in both complete information setting and 
the incomplete one. A very interesting problem in the complete information case is to generalize 
our model to a setting where stacking and retrieving processes overlap in time. In such a setting, 
the goal would be to jointly optimize the number of relocation for stacking and retrieving. We 
believe that one of the major questions of this problem is how to prioritize between stacking and 
retrieving at a given time. In the incomplete information case, one can easily generalize our two- 
stage stochastic optimization problem to a multi-stage setting where ASA* could be extended. 
However, as we increase the number of stages, the computation time grows rapidly. Additional 
assumptions and/or other efficient heuristics would be needed to ensure a reasonable running time. 
Finally, another interesting problem is to study the case where no information about any containers 
is known in advance. Intuitively, it appears that the policy of placing the container on the emptiest 
column should be the optimal solution; however this remains to be proven. 

Other storage systems such as steel plate stacking and warehousiM systems face the relocation 
problem with complete and incomplete information (see Kim et al. jj, Zapfel and Wasner [l^ for 
the former, and Chen et al. 0| for the latter). We believe that the frameworks proposed in this 
paper could also be applicable to those problems. 
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Appendix 


Proof of Provosition [27M Between level I and 1 + 1, one move has been operated (r moved to 
column c,). If r < min(C;) (‘good” move) then r was a blocking container in but not in thus 
So{B^^^) = So(B^) - 1. If r > min(Q), we have So{B^^^) = So{B^)- So So{B^^^) = So(B^) - xi^ < iriin(cO) 
where x is the indicator function. Therefore Lo{B^'''^) = So{B^'''^) + (/ + !) = So(B^) - xi^ < rnin(c!)) + 
1 + / > So(B') + I = Lo(B')- □ 

Proof of Proposition 12-41 We have Lp(B) - Lp-i(B) = L/-e«p(B) tC?" > MM(Bp)). Since is a non 
negative function, the results follows. □ 

Proof of Provosition \2. 5l Intuitively, this property comes from the fact that Sp{B^'^^) > Sp{B^) — 
1. Indeed, if this holds, we have 

Lp(B'+i) = Sp(B'+i) + / + 1 > Sp(B') -! + / + ! = Sp(B) + I = Lp(B) 

So let us prove Sp{B^'^^) > Sp{B^) — 1. Let r be the container that is relocated between B^ and B^"*"^. 
We have two cases: 

• If r can do a good move, r has only been counted once in Sp(B^) as it is a blocking container 
in B^ and it might or might not contribute to Sp{B^'^^). Therefore Sp(B^'^^) > Sp{B^) — 1. 

• If r cannot do a good move, it has been counted twice in Sp{B^). But since it cannot do a 
good move, it is necessarily a blocking container in so it counts at least once in Sp{B^'^^). 
Thus Sp(B'+i) > Sp(B') - 1. 

This completes the proof. □ 

Proof of Proposition [^77[ Using heuristic H, a container is relocated only if it is blocking. For 
any blocking container, there exists at least one empty column which means there is a possible 
good move. So each blocking container is relocated at most once, i.e, Zh{B) = So(B). Since So(B) < 
Zopt(B) < Zh{B) = Sq{B) and we have equation dH). □ 

Proof of Proposition I^.Bl For simplicity of proofs we assume that ni , the target container in 
the bay is 1. If 1 is not blocked then it is retrieved. In that case, there are C containers left and 
thus we use Proposition 12.71 (note that in that case Si(B) = So(B)). 

Then let us suppose 1 is blocked by some containers. If there is more than one blocking 
container, all except the last one can be relocated to empty columns. So we come back to the case 
where 1 is only blocked by container r, and there is no empty column. 

Now we consider two cases: If r can do a good move and H relocates r to a good column and then 
retrieves 1. We call B' the new configuration. By Proposition [2171 we have Zh{B') = So(B') = Si(B'). 
Notice that Zh(B) = Zh(B'} + 1 and Si(B) = Si(B') + 1 and the result follows. 
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Now suppose r cannot do a good move. This can only happen if r = C + 1 and all the other 

columns have exactly one container. It is easy to check that in that case, Zh(B) = Si(B). □ 

Proof of Provosition f^TPl First let us introduce some notations. Let B be the initial configura¬ 
tion and B' be the configuration obtained after retrieving container 1 using heuristic H. We denote 
Ri the set of containers blocking container 1 in B. 

Now let us consider the case k = 2. Note that using Proposition 12.81 we have Zh(B') = Si(B') 
and Si(B') < So(B') -I-1. There are four possibilities to consider: 

• If |Ri| = 0, then it is easy to see that Zh{B) = Zh{B') = Zopt{B') = Zopt{B). 

• If |Ri| = 1, we know that So(B') < So(B). Therefore Zh(B) = 1-1- Zh{B') = 1-1- Si(B') < 

1 + So(B') + 1 < So(B) + 2 < ZoptiB) + 2. 

• If |Ri| > 2, then there are at least |Ri| — 2 columns that are empty in B. Therefore the |Ri| — 2 
topmost containers blocking 1 find necessarily a ’’good” column in B. Hence this case can be 
reduced to the case where |Ri| = 2. 

• If Rj = 2, then we claim that Si(B') < So(B). If this is true, then Zh(B) = Zh(B') + 2 = 
Si(B') + 2 < So(B) -I- 2 < ZoptiB) + 2. Now let us denote ri and r 2 the two containers blocking 1 
in B. 


— If ri < C or r 2 < C then So(B') < So(B) - 1 and therefore Si(B') < So(B') -I- 1 < So(B). 

— If ri > C and r 2 > C, then if one of them finds an empty column then again So(B') < 
So(B) —1. Otherwise it means that all columns have one container and C-l-1 and C-l-2 are 
on top of 1. In those two bays we can verify that Zh(B) = 4 < 2 -I- 2 = So(B) + 2 < Zgpt + 2. 

This concludes the case k = 2. 

For 3 < k < C, the proof is simpler and works by induction. For fc = 2, the previous paragraph 
proves a stronger inequality. Let us suppose it is true for fc - 1, and B has C + k containers. We 
have by induction hypothesis, Zh(B') < So(B') + We have three cases: 

. If |Ri| = 0, then zh(B) = zh(B') < So(B') + ^ < So(B) + ^ < Zopt(B) + 

• If |Ri| > k, then there are at least (C - 1) - (C + k - (|Ri| -I-1)) = |Ri| - B empty columns and 

therefore So(B') < So(B) + k- |Ri| which implies Zh(B) = |Ri| -I- Zh(B') < |Ri| -I- Sq{B') + < 

|Ril + So(B) + k- |Ri| + ^ = So{B) + ^ < ZoptiB) + 

• If 0 < |Ri| < /c, then since So(B') < So(B) we have Zh(B) = |Ri| -I- ZniB') < |Ri| -I- So(B') -I- < 

k + So(B) + ^ = So(B) + < ZoptiB) + 

which concludes the proof. 

□ 
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Proof of Provosition [3771 Let Sg(Bc) be the number of blocking containers in column t. By the 
linearity of expectation, we have 


Ec [So(Bc)] = Ec 


■ c 


!=1 


C 

= ^^e[Sq(Bc)] = a;, X 
i=l 


C, 


where af, = E|^Sg(Bc)J. The last equality comes from the fact that each column is identically 
distributed. 

Now let us compute a^. By definition, ajj = kpk,h- Let us start with po,^- Given h randomly 
chosen containers, the probability that they are placed in the first column in a descending order 
(meaning there is no blocking container) is po,/! = Now for 1 < k < h — 1, we compute by 
recursion (conditioning on the position of the smallest container). Let n* be the smallest container 
among these h containers, n* is located in topmost tier with probability Conditioned on that 
event, it means that (} — 1) containers are blocking n*. So if / > k + 1, then the event of having 
k blocking container is not possible. Therefore for j < k + 1 there should be (k — {j — 1)) blocking 
containers below the topmost tier and this happens with probability pk-{j-i),h-j (since there are 
h — i tier below the topmost tier). Summing over all j (possible positions of n*), we have 



1 


which completes the proof □ 

Proof of Theorem \3.3\ . Since for all configurations B^, ZgptiBc) > So(Bc) then ^ 1- 

Moreover, we have: 


where 


Ec [zopt(Bc)] ^ Ec [zopf(Bc)] - Ec [So(Bc)] 
Ec[So(Bc)] Ec [So(Bc)] 


= 1 + ^ (Ec [zopt(Bc)] - ahC) 


= 1 + 


UhC 


(14) 


g{C) = Ec [zopt(Bc)] - apC. 


(15) 


In Lemma l6.11 we study how Ec |^Zopf(Bc)] evolves and we show that it increases almost linearly in 
a which shows that the function g(.) is essentially bounded. 


Lemma 6.1. Let Zopt(Bc) be the optimal number of relocations for configuration Be, then there 
exists a constant 9 (defined in equation f3l^) such that: 


Ec+i [zopf(Bc+i)] < Ec [zoptiBc)] + ah + h(P - 1)(C + l)c-®(C+i)^ VC > B + 1 (16) 
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Using Lemma l6.ll we have for all C >h + 1: 


Ec+i [zopf(Bc+i)] ^ Ec [zopf(Bc)] + o:h + h{P - 1)(C + l)e 
Ec+i [zopt{Bc+i)] - aniC + 1) < Ec [zoptiBc)] - ochC + h{P - 1)(C + l)e-®(C+i) 
^ g{C + 1) < g{C) + h{P - 1)(C + 

C OO 

g{C) < g(h + 1) + h(P - 1) ^ (/h-®') < g(h + 1) + h(P -1)Y^ 

i=h+2 i=l 


=> §-(0) < g{h + 1) + 


e^hjP - 1 ) 


= K'. 


Therefore using equations (fTT|) and (fTTll . we have 


Ec [zopf(Bc)] K 
Ec[So(Bc)] 


= /(C), 


where 



a 

which completes the proof of the theorem. 


g{h + 1) + 


e<^h{P-l) 

{eO-lf 


a 


(17) 


(18) 


(19) 

□ 


Proof of Lemma \6.1i Now we need to prove equation (jl6p . Define a column to be “special” if 
all its containers are not smaller than co = {h — 1)(C +1) +1. Now let us consider the following event: 

Q = I The bay with C + 1 columns has at least one “special” column |. (20) 

The intuition is the following: the probability of having a “special” column grows quickly to 1 
as a function of C implying that the event Q happens with high probability. Now, conditioned on 
Q, we more easily express the difference between bays of size C + 1 and C in the following way. We 
claim that 


Ec+i [zopf(Bc+i)|0] < Ec [zopf(Bc)] + ap, (21) 

Let Bc+i be a given bay of size C + 1 that verifies Q. Since columns in bays can be interchanged, 
we suppose that a “special” column is the first (leftmost) column of the bay. We also denote 
ni,n 2 ,... rfip the containers of the first column. We know that ni,n 2 , ■ ■ ■ ,np > co and tii n 2 ^ 
... t np. Finally let Bq be the bay Bc+i without its first column (see Figure [15]). 

First we prove that 

Zopt(Bc+l) — Zopf(Bc) + So 


ni 

Mp 


( 22 ) 
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Be 



Figure 15: Bay decomposition of Bc+i (The part on the right has C columns) 


To prove equation (1221) . we construct a feasible sequence o for the bay of size C + 1 for which the 
number of relocations is equal to the right side of equation Let Oopt{Bc) the optimal sequence 
for Bq, t' be the first time step when the target container in Oopt{Bc) is larger than min{ni, n 2 , ■ ■ ■, Tip] 
and B'^ be the bay obtained at t' using Oopt{Bc)- Let the first f' — 1 moves of a be the first f' — 1 
moves of Oopt{Bc)- Note that B'^ has at most C-h containers due to the choice of oj. By Proposition 
the number of relocations performed by Oopt{Bc) from t' until the end is So{B'^). Therefore 


ZoptiBc) = # relocations up to t' done by ZgptiBc) + So{B'^). 

Ill 


(23) 


After t', we run heuristic H on B' 


c+i 


tip 


U B' 


We claim that is exactly the right side of equation (I22p . There are at most C containers in 
B^^^, therefore using ProDosition l2.7l we know that if we apply the heuristic H to this configuration, 

"" ni 

then the number of relocations done by H is So(B^^^) = So(B^) + So ... . Therefore 


np 


Zopt(Bc+i) ^ Zo{Bc+i) = # relocations up to t' done by Zopt{Bc) + So(B^) + So 


ni 

np 


\i « j/ 


and we have 


Zopt(Bc+i) — z^optiBc) “t So 


ni 

np 


\i « j/ 


(24) 


which proves equation 


Now we can take the expectation from both sides of equation (|21l) over a uniform distribution 
of the rest of the hC containers that are not in the first column. We claim that the first term on 
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the right hand-side of equation is exactly Ec [zopf(B)|. For any configuration that appears in 
Be, we can map it to a unique configuration where all containers are between 1 and hC. Thus, 


Ec 

Zopf (Bc+i) 

Ml 



. [ 


< 


Ec [zopf(®c)] + So 


ni 


fih 


(25) 


Next, we take the expectation of both sides of equation ([2^ over possible first columns, which 
is a “special” column. Now notice that if Bc+i is generated uniformly in the sets of bays of size 
C -t 1, then conditioned on Q, the probability of having a certain column [ni,.. is identical 

for any n\ + ... + nh> a and it is given by 


P 


ni 




Q 


(C-h)! 
C! ■ 


(26) 


Moreover, if we withdraw this first column from Bc+i look at the last C columns (after renum¬ 
bering the containers so that the numbers lie between 1 and hC), then this bay is also uniformly 
distributed in the sets of bays with C columns. Therefore we can write: 


Ec+i [zopt(Bc+i)io] = y, 


rij '' 
«,■> CO 


Ec 


= E 


{ni,-,nh) 
ni+ tij 
ni> (o 


Ec 


Zopt(®C+l) 

Ml 

- 

rih 

- 

n\ 

Zopt(®C+l) 



nh 


,Q 


X P 


X P 


ni 

rih 


ni 

nh 

Q 


Q 


(27) 


(28) 



f 

Ml 

\ 


/ 

Ml 

\ 

/ 

Ml 

\ 

< Ec[zopt{Bc)] y E 



Q 

+ y So 



X P 



Q 

ni+ rij 


np 


nii= rij 

1 

Up 

/ 


np 

/ 


«i> (O 


^ Ec [zopt(Sc)] + y, Sq 


m* Hj 
ni> CO 


ni 


fih 


ni> CO 
/ 


X P 


ni 

Hh 


Q 


(29) 


The equality between 1271 and 1251 comes from the fact that if we know that Bc+i has a “special” col 

ni 


umn, then we do not need to condition on Q. Equation[22]uses the fact that E I’ 

1 . 


rtii^ nj, 

n:> CO 


nh 


Q 
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Note that, given any (ni,... ,n;,) such that n, + rij, we have 


ni 






So 


(Xh) 


u j; 


when the expectation is over a random order of (ni,..., n/j). This is true regardless of the set 
that is drawn from (See Remark 13.2p . This implies that the second term in the right 
hand side of equation (|2^ is equal to a/,; Therefore, we get equation (|2T|) . 

Lemma 16.21 states that the event Q has a probability that increases exponentially fast to 1 as a 
function of C. 


Lemma 6.2. Let Q he the event defined by equation h2(^) . then there exists a constant 6 > 0 such 
that 


P(Q) < 


(30) 


where 0 is given by equation 

Now we want to focus on the event Q. We give an upper bound on E^+i |^Zopt(Bc+i)|0]. For any 
conhguration, in order to retrieve one container, we need at most P — 1 relocations (since at most 
P — 1 containers are blocking it), thus for any configuration, the optimal number of relocations is 
at most P — 1 times the number of containers (h(C + 1)) which gives us h{P — 1)(C + 1) as an upper 
bound on the optimal number of relocations. We use this universal bound to get 

Ec+i [zopf(Bc+i)|Q] < h{P - 1)(C + 1). (31) 

Finally using Lemma 16.21 we have 

Ec+i [zopt(Bc+i)] = Ec+i [zopt(Bc+i)|o]P(0) + Ec+i [zopt(Bc+i)|o] P(0) 

< Ec+i [zopKBc+i)|o] + Ec+i [zopKBc+i)|o] x 

< Ec [zoptiBc)] + ah + HP - 1)(C + 

which completes the proof of Lemma 16.11 □ 

Proof of Lemma IB. 21 Recall that 

Q = I The bay with C + 1 columns has at least one “special” column |. 

We know that each bay of size C + 1 can be mapped to a permutation n of iS;;(c+i) taken 
uniformly at random. Let qf) be the function from S^c+i) to defined by 

q : n i—> number of “special” columns in the resulting bay of n. 
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Note that 


p(q) = P(^j(7t) = 0). 


First we compute the expected value of q{.) 

rc+i 


Ec+i[^l] - Ec+1 


is a “special” column) 
i=l 


Ec+i[^l] = (C + 1) X P {{ci is a “special” column}), 

where we use linearity of expectation and the fact that columns are identically distributed. 
A simple counting implies that 0: 

(C + l)[(C + l)-l]...[(C + l)-h + l] 


( 32 ) 


P ({ci is a “special” column]) 


> 


hiC + l)[h{C + 1) - 1]... [hiC + l)-h + l] 

{C + l)-h + l ’ 
h{C +1) 

\h 


> 


h{h + 1) ' 


(33) 


where we use C + l>h + lto show the last inequality. 


Therefore we know that 

Ec..WMC+l)x(^)''. (34) 

We claim that (^(.) is well concentrated around its mean. In order to do so, we prove that q{.) 
is 1-Lipschitz. Define p the distance between two permutations 7 Ti,7Z2 € 'S^c+i) ^ pi'^i^'^i) = 
I {i € [h{C + 1)] : TTi(z) + 712(011- We want to prove that 


\qini) - q{n 2 )\ < p{ni, m), V (tzi, m) e Sh{c+i)- 


(35) 


Let Til, 712 £ Let us first consider the case where p(7Ti,7i2) = 2. (Notice that if p(7Zi,7Z2) + 0 

then p(7Zi,7T2) > 2). In that case, we have i,] e such that 7Ti(0 = ’^lij) and 7 Ti(/) = 712(0- 

Let Bi and B 2 be the configurations generated by tzi and 7Z2. Having p(7Zi, 7 Z 2 ) = 2 corresponds to 
the fact that if we swap 2 containers in Bi, we get B 2 and we denote those containers a = 7Zi(0 and 
b = 7Zi(/). We have three cases: 


a and b are both in “special” columns in Bi. In this case, swapping them will not change 
anything since both their new columns in B 2 will also be “special” and hence \q{Tii)—q{n 2 )\ = 0. 


^Notice that when C —> 00 , the probability is equivalent to (-)* which would guarantee a faster convergence rate. 
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• a and h are both in columns that are not “special” columns in Bi. li a,h > oj ox a,b < co then 
we do not create any new special column in B 2 . Now suppose that a> co and b < o), then the 
column of a in B 2 might be a “special” column, but the column of b in B 2 cannot be “special”. 
Therefore in that case, \c]{ni) - Cj{n 2 )\ < 1- 

• fl is in a “special” column in Bi but b is not. Now we know that a > a>. If b < a> then 
the column of b in B 2 cannot be “special” but the column of a might be and in that case 
l^(^i) “ '?(^2)l ^1- If b > 0 ), then the column of b in B 2 is “special” and the column of a in B 2 
is not “special” which gives us \cj{Tii) - q{n 2 )\ = 0. Note that the proof is identical if is in a 
“special” column in Bi but a is not. 

So far we have shown that 


If pint, 712 ) = 2, then \q{ni) - q{n 2 )\ < 1. 


(36) 


Now we suppose that p{ni, 712 ) = k where 2 < k < h{C + 1). Note that we can construct a 
sequence of permutations (ripTi',.. such that 71 ^ = rci, 7z(. = 7 x 2 and p(7zl,= 2. 

Now using this fact and equation (f36]l . 


\q{ni) - q{n 2 )\ 


k-l 


k-l 


k-l 


q{n\) - ^?(7T'+l) < |^?(7i') - (?(7l'+i)| <J^l = k-l<k = pim, 712), 

i=l i=l i=l 


which proves that q{.) is 1-Lipschitz. 


Now we use Theorem 8.3.3 of Matousek and Vondrak 


181 which states that P ((? < Ec+i [^] “ 0 - 


_ r 

e 8''(c+i) and apply it with t = Ec+i [^] and equation pll) to get 


F{q = 0) = F{q< Ec+i [q] - Ec+i [q]) < e ^kc+d 


< g-e(c+i)^ 


where 



h{h + 1) 


2h 


> 0 , 


(37) 


which concludes the proof. 

Proof of Corollary Using that So(Bc) < ZgeniBc) ^ Zopf{Bc), we have 1 
^dSolBctl^ - /(^)- Finally we know that Ec [So(Bc)] = a;,C. 

Proof of Proposition \4.1\ We can write E[ei] as follows: 

E[ei] = (E[E[Zp^s^] - - (E[E[Zp.] - Zp.]) + F[Zp^^^ - Zp^] < E[|E[Zp^s^] - Zp^,^ 


□ 


^ ]Ec[z^cti(Sc)] ^ 

- EclSolBc)] - 
□ 


|] + E[|E[Zp.] -Zp.|].(38) 
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The above inequality holds because ASA* chose Pasa, and thus we have < Zp*. Moreover, 
we can compute E[|E[Zp^ 5 ^]-Zp^ 5 ^|] and E[|E[Zp*]-Zp»|] using their CDF. We denote |E[Zp^ 5 ^]-Zp^g^| 
by A. Note that 0 < A < r^ax and inequality (fTTD gives a bound on its CDF, when estimating Zp^g^ 
with S samples. For E[A], we have: 


E[A] = 


Jo 

^ i 

r 

= ^ 

Jo 


1 - FA(x)dx 

„ ,-2Sx2 

2 exp(^- )dx 

^ max 

nax ^ £ r°° £ ^ 

{-)6^dx<2 (-)i 


s^dx 



(39) 

(40) 

(41) 


We can compute E[|E[Zp*] - Zp*|] in a similar way. 

The first equality in (1411) is obtained by substituting S with the expression in (11211 . Using the 
bound on E[A] and inequality (l38l) . we can compute E[ei] as follows: 


E[Ci] < 2S (42) 

i-ln(-) 

□ 

Proof of Proposition For an arbitrary initial bay B, let p be the path with the minimum 
estimated upper bound. Notice that pruning p may be a mistake if the true mean of Lp is less 
than the true mean of Up, i.e., E[Lp] < E[Up]; in such a case, £2 < E[Llp] - E[Zp] < E[Up] - E[Lp]. 
Otherwise £2 is zero; thus we have £2 < (E[Up] - E[Lp])'''. 

Let us denote Lp — E[Lp] and E[Llp] - Up by Xi and Xu, respectively. Also, let d he Lp — Up. 
The true and estimated values of the upper bound and lower bound, and the loss are illustrated in 
Figured £2 is shown by the thick line segment. For any bay with an arbitrary initial configuration, 
we can bound £2 as follows: 

£2 < ((E[lip] - Up) + (Up - Lp) + (Lp - EILp]))"^ = (xu-d + Xl)^. (43) 

The expected loss, E[e 2 ], can be bounded as shown in ()i6]l . 


E[e2] = E[(xl + Xu- d)+] 


< 

< 

< 


E[(xl + xu)+] < E[|xl + xu|] 
E[|xl| + |xu|] = E[|xl|] + E[|xu|] 
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(44) 

(45) 

(46) 











The first inequality in [H] holds because d is always positive. Inequality 05] results from the 
triangular inequality, and the last inequality is obtained by replacing E[|x£|] and E[|xLf|] by the 
expression in (1411) . 

□ 


Proof of Proposition [4^ For an arbitrary initial bay B, let pmax be the path that maximizes 
E[LIp] among pi,... ,pm. We bound e$ (the expected loss due to prununig) as follows: 


63 < P(mistake at fi U • • • U mistake at tm)(E[zp^^] - E[Zp*]) (47) 

< m P(mistake at fi)(E[LIp^„^] - 0) (48) 

= m P(mistake at ti){E[Up„,J - Up^^ + Up^J (49) 

^~THCIX 

< m P(mistake at fi)(|X[i| + Up ). (50) 


Notice that inequality 08] is obtained by using the union bound. Moreover, we replace the 
probability of mistakes at each step by the maximum probability. Without loss of generality we 
assume that maximum probability is at f^. Also note that we do not know which path is pmax 
because we have not observed E[LIpJ,.. .,E[LIp^]. Nevertheless, regardless of the path, we can 
replace - Up^^ with |X(j| (by definition) to obtain (150]) . Also, we can bound Up^^^ by Up . 

Now we compute P(mistake at U). Let us denote |xl| and by x and y, respectively. Also, 
let a; = X + y and /w(w) be the PDF of zv. Notice that the probability of making a mistake depends 
on the value of d; the maximum probability corresponds to the stage with the smallest value of d 
and can be computed as follows: 


J r*oo ^00 pzv 

fw{w)dw = I I fx{w - y)fY{y) dy dw. (51) 

d-min dmiy, 


/x(x) and friy) can be obtained from (fTTI) . and the above integral can be bounded as follows: 


e (P+(w-y)^) 


fx{w - y)fY{y) = 4( ) y{w - y)(-) ^>2 

Jdmin do ddmin d0 ^ 


''‘■min 

w2 


dn 


= (£)%" +HIE'’ 

^ 2 ’ 6 


6 -mm dmin -yj ^^^( 2 ). 
—- (- 2 s 2 Erf(---) 

2 2 V 26 


dn 


- ^ 2 ’ 6 




( 2 ) 


z? ■ 
t mm 

' ,2 


(52) 


From [ 50 I [ 5 ^ and expression 0T] for E[|x(j|], it follows that: 


7 

€ ^ wm 


E[e3] < m (2)ir + 2= ^ 


-ln{^)7i 


7''*" £ “min\/ 


TT —max 

e7 


■ln(2) 


)■ 


^2' ' 6 \ 2 
where Up = max{Up^,..., Up^}, and dmin = nrin{di, ... ,dm}- 


(53) 
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